Evaluation of reproducibility of protein identification results after multidimensional human serum protein separation

We developed a gel- and label-free proteomics platform for comparative studies of human serum. The method involves the depletion of the six most abundant proteins, protein fractionation by Off-Gel IEF and RP-HPLC, followed by tryptic digestion, LC-MS/MS, protein identification, and relative quantification using probabilistic peptide match score summation (PMSS). We evaluated performance and reproducibility of the complete platform and the individual dimensions, by using chromatograms of the RP-HPLC runs, PMSS based abundance scores and abundance distributions as objective endpoints. We were interested if a relationship exists between the quantity ratio and the PMSS score ratio. The complete analysis was performed four times with two sets of serum samples containing different concentrations of spiked bovine beta-lactoglobulin (0.1 and 0.3%, w/w). The two concentrations resulted in significantly differing PMSS scores when compared to the variability in PMSS scores of all other protein identifications. We identified 196 proteins, of which 116 were identified four times in corresponding fractions whereof 73 qualified for relative quantification. Finally, we characterized the PMSS based protein abundance distributions with respect to the two dimensions of fractionation and discussed some interesting patterns representing discrete isoforms. We conclude that combination of Off-Gel electrophoresis (OGE) and HPLC is a reproducible protein fractionation technique, that PMSS is applicable for relative quantification, that the number of quantifiable proteins is always smaller than the number of identified proteins and that reproducibility of protein identifications should supplement probabilistic acceptance criteria.

Loss on ignition as a method for estimating organic and carbonate content in sediments: reproducibility and comparability of results

Five test runs were performed to assess possible bias when performing the loss on ignition (LOI) method to estimate organic matter and carbonate content of lake sediments. An accurate and stable weight loss was achieved after 2 h of burning pure CaCO3 at 950 °C, whereas LOI of pure graphite at 530 °C showed a direct relation to sample size and exposure time, with only 40-70% of the possible weight loss reached after 2 h of exposure and smaller samples losing weight faster than larger ones. Experiments with a standardised lake sediment revealed a strong initial weight loss at 550 °C, but samples continued to lose weight at a slow rate at exposure of up to 64 h, which was likely the effect of loss of volatile salts, structural water of clay minerals or metal oxides, or of inorganic carbon after the initial burning of organic matter. A further test-run revealed that at 550 °C samples in the centre of the furnace lost more weight than marginal samples. At 950 °C this pattern was still apparent but the differences became negligible. Again, LOI was dependent on sample size. An analytical LOI quality control experiment including ten different laboratories was carried out using each laboratory's own LOI procedure as well as a standardised LOI procedure to analyse three different sediments. The range of LOI values between laboratories measured at 550 °C was generally larger when each laboratory used its own method than when using the standard method. This was similar for 950 °C, although the range of values tended to be smaller. The within-laboratory range of LOI measurements for a given sediment was generally small. Comparisons of the results of the individual and the standardised method suggest that there is a laboratory-specific pattern in the results, probably due to differences in laboratory equipment and/or handling that could not be eliminated by standardising the LOI procedure. Factors such as sample size, exposure time, position of samples in the furnace and the laboratory measuring affected LOI results, with LOI at 550 °C being more susceptible to these factors than LOI at 950 °C. We, therefore, recommend analysts to be consistent in the LOI method used in relation to the ignition temperatures, exposure times, and the sample size and to include information on these three parameters when referring to the method.

Effect of Population Heterogenization on the Reproducibility of Mouse Behavior: A Multi-Laboratory Study

In animal experiments, animals, husbandry and test procedures are traditionally standardized to maximize test sensitivity and minimize animal use, assuming that this will also guarantee reproducibility. However, by reducing within-experiment variation, standardization may limit inference to the specific experimental conditions. Indeed, we have recently shown in mice that standardization may generate spurious results in behavioral tests, accounting for poor reproducibility, and that this can be avoided by population heterogenization through systematic variation of experimental conditions. Here, we examined whether a simple form of heterogenization effectively improves reproducibility of test results in a multi-laboratory situation. Each of six laboratories independently ordered 64 female mice of two inbred strains (C57BL/6NCrl, DBA/2NCrl) and examined them for strain differences in five commonly used behavioral tests under two different experimental designs. In the standardized design, experimental conditions were standardized as much as possible in each laboratory, while they were systematically varied with respect to the animals' test age and cage enrichment in the heterogenized design. Although heterogenization tended to improve reproducibility by increasing within-experiment variation relative to between-experiment variation, the effect was too weak to account for the large variation between laboratories. However, our findings confirm the potential of systematic heterogenization for improving reproducibility of animal experiments and highlight the need for effective and practicable heterogenization strategies.

Diagnostic reproducibility of tumour budding in colorectal cancer: a multicentre, multinational study using virtual microscopy

Puppa G, Senore C, Sheahan K, Vieth M, Lugli A, Zlobec I, Pecori S, Wang L M, Langner C, Mitomi H, Nakamura T, Watanabe M, Ueno H, Chasle J, Conley S A, Herlin P, Lauwers G Y & Risio M (2012) Histopathology Diagnostic reproducibility of tumour budding in colorectal cancer: a multicentre, multinational study using virtual microscopy Aims:  Despite the established prognostic relevance of tumour budding in colorectal cancer, the reproducibility of the methods reported for its assessment has not yet been determined, limiting its use and reporting in routine pathology practice. Methods and results:  A morphometric system within telepathology was devised to evaluate the reproducibility of the various methods published for the assessment of tumour budding in colorectal cancer. Five methods were selected to evaluate the diagnostic reproducibility among 10 investigators, using haematoxylin and eosin (H&E) and AE1-3 cytokeratin-immunostained, whole-slide digital scans from 50 pT1-pT4 colorectal cancers. The overall interobserver agreement was fair for all methods, and increased to moderate for pT1 cancers. The intraobserver agreement was also fair for all methods and moderate for pT1 cancers. Agreement was dependent on the participants' experience with tumour budding reporting and performance time. Cytokeratin immunohistochemistry detected a higher percentage of tumour budding-positive cases with all methods compared to H&E-stained slides, but did not influence agreement levels. Conclusions:  An overall fair level of diagnostic agreement for tumour budding in colorectal cancer was demonstrated, which was significantly higher in early cancer and among experienced gastrointestinal pathologists. Cytokeratin immunostaining facilitated detection of budding cancer cells, but did not result in improved interobserver agreement.

Intrapatient reproducibility of the BISxp monitor

BACKGROUND: The Bispectral Index (BIS) reportedly reflects anesthetic depth. It is recommended that anesthetic agents should be titrated to maintain the BIS between 40 and 60 arbitrary BIS units during anesthesia. For anesthesia providers to follow this recommendation, the monitor should be predictably affected by different anesthetic agents and have good interpatient and intrapatient reproducibility. The authors hypothesized that when two BISxp devices (Aspect Medical Systems, Newton, MA) are placed concurrently on the same patient, their readings are concordant throughout the anesthetic period. METHODS: Simultaneous BIS recordings from two BISxp monitors were obtained during anesthesia at 5-s intervals from 12 participants. RESULTS: In total 22,860 concurrent paired BIS readings were obtained. For 10.7% of the time, there were sustained periods of 30 s or greater where the readings suggested a different depth of anesthesia. For 6% of the time, there were sustained periods of 30 s or greater where the readings differed by 10 or more arbitrary BIS units. The regression coefficient (R) for the two devices was 0.65 (range, 0.35-0.92). There was zero bias between the devices, and the 95% limits of agreement ranged between -18 and +17. CONCLUSION: A conflicting anesthetic management was suggested by the simultaneous BIS readings 10.7% of the time. These results suggest that BISxp does not always provide a reproducible single number. Anesthesia providers should not rely exclusively on the BIS reading when assessing depth of anesthesia.

Reproducibility of dGEMRIC in assessment of hip joint cartilage: a prospective study

PURPOSE: To investigate the reproducibility of dGEMRIC in the assessment of cartilage health of the adult asymptomatic hip joint. MATERIALS AND METHODS: Fifteen asymptomatic volunteers (mean age, 26.3 years +/- 3.0) were preliminarily studied. Any volunteer that was incidentally diagnosed with damaged cartilage on MRI (n = 5) was excluded. Ten patients that had no evidence of prior cartilage damage (mean age, 26.2 years +/- 3.4) were evaluated further in this study. The reproducibility of dGEMRIC was assessed with two T1(Gd) exams performed 4 weeks apart in these volunteers. The protocol involved an initial standard MRI to confirm healthy cartilage, which was then followed by dGEMRIC. The second scan included only the repeat dGEMRIC. Region of interest (ROI) analyses for T1(Gd)-measurement was performed in seven radial reformats. Statistical analysis included the student's t-test and intra-class correlation (ICC) measurement to assess reproducibility. RESULTS: Overall 70 ROIs were studied. Mean cartilage T1(Gd) values at various loci ranged from 560.9 ms to 684.4 ms at the first set of readings and 551.5 ms to 662.2 ms in the second one. The mean difference per region of interest between the two T1(Gd)-measurements ranged from 21.4 ms (3.7%) to 45.0 ms (6.8%), which was not found to be statistically significant (P = 0.153). There was a high reproducibility detected (ICC range, 0.667-0.915). Intra- and Inter-observer analyses proved a high agreement for T1(Gd) assessment (0.973 and 0.932). CONCLUSION: We found dGEMRIC to be a reliable tool in the assessment of cartilage health status in adult hip joints.

Variability of anti-PF4/heparin antibody results obtained by the rapid testing system ID-H/PF4-PaGIA

BACKGROUND: Recent studies have shown that a low clinical pretest probability may be adequate for excluding heparin-induced thrombocytopenia. However, for patients with intermediate or high pretest probability, laboratory testing is essential for confirming or refuting the diagnosis. Rapid assessment of anti-PF4/heparin-antibodies may assist clinical decision-making. OBJECTIVES: To evaluate the performance of rapid ID-H/PF4-PaGIA. In particular, we verified reproducibility of results between plasma and serum specimens, between fresh and frozen samples, and between different ID-H/PF4-polymer lots (polystyrene beads coated with heparin/PF4-complexes). PATIENTS/METHODS: The samples studied were 1376 plasma and 914 corresponding serum samples from patients investigated for suspected heparin-induced thrombocytopenia between January 2000 and October 2008. Anti-PF4/heparin-antibodies were assessed by ID-H/PF4-PaGIA, commercially available ELISAs and heparin-induced platelet aggregation test. RESULTS: Among 914 paired plasma/serum samples we noted discordant results (negative vs. low-titre positive) in nine instances (1%; 95%CI, 0.4-1.6%). Overall, agreement between titres assessed in plasma vs. serum was highly significant (Spearman correlation coefficient, 0.975; P < 0.0001). Forty-seven samples tested both fresh and after freezing/thawing showed a good agreement, with one discordant positive/negative result (Spearman correlation coefficient, 0.970; P < 0.0001). Among 1376 plasma samples we noted a strikingly variable incidence of false negative results (none - 82%; 95%CI, 66-98%), depending on the employed ID-H/PF4-polymer lot. Faulty lots can be recognized by titrating commercial positive controls and stored samples of HIT-patients. CONCLUSION: Laboratories performing the assay should implement stringent internal quality controls in order to recognize potentially faulty ID-H/PF4-polymer lots, thus avoiding false negative results.

Reproducibility of cerebral phenylalanine levels in patients with phenylketonuria determined by 1H-MR spectroscopy

The reproducibility of metabolite content determined by MR spectroscopy (MRS) is usually at best a few percent for the prominent singlets. When studying low-concentration metabolites, like phenylalanine (Phe), where tissue content can be <100 micromol/kg, better reproducibility is paramount-particularly in view of using MRS results for potential individual treatment advice. An optimized, targeted spectroscopy method was established at 1.5T and reproducibility was established in 21 patients with phenylketonuria (PKU) where three spectra were recorded in each of three independent sessions, two of which were in immediate succession to minimize physiologic variation. Intersession variation was found to be only 7 micromol/kg Phe for back-to-back repetition of sessions, in close agreement with the variation of 16 micromol/kg observed for single spectra within a session. Analysis of variance proved the individuality of the blood/brain Phe ratio-though this ratio seems to be influenced by physiologic factors that are not stable in time. The excellent reproducibility was achieved through optimization of various factors, including signal-to-noise ratio, repositioning, and prescan calibrations, but also by enforcing as much prior information as possible (e.g., lineshape and phase from reference scans, constant prior-knowledge-locked baseline). While the application of maximum general prior knowledge is a general method to reduce fluctuations, one should remember that it may introduce systematic errors.

Reproducibility of retinal thickness measurements in healthy subjects using spectralis optical coherence tomography

PURPOSE: To test the reproducibility of retinal thickness measurements in healthy volunteers of a new Frequency-domain optical coherence tomography (OCT) device (Spectralis OCT; Heidelberg Engineering, Heidelberg, Germany). DESIGN: Prospective, observational study. METHODS: Forty-one eyes of 41 healthy subjects were included into the study. Intraobserver reproducibility was tested with 20 x 15 degree raster scans consisting of 37 high-resolution line scans that were repeated three times by one examiner (M.N.M.). Mean retinal thickness was calculated for nine areas corresponding to the Early Treatment Diabetic Retinopathy Study (ETDRS) areas. Coefficients of variation (COV) were calculated. RESULTS: Retinal thickness measurements were highly reproducible for all ETDRS areas. Mean total retinal thickness was 342 +/- 15 microm. Mean foveal thickness was 286 +/- 17 microm. COVs ranged from 0.38% to 0.86%. Lowest COV was found for the temporal outer ETDRS area (area 7; COV, 0.38%). Highest COV was found for the temporal inner ETDRS area (area 3; COV, 0.86%). Mean difference between measurement 1 and 2, measurement 1 and 3, and measurement 2 and 3 for all ETDRS areas was 1.01 microm, 0.98 microm, and 0.99 microm, respectively. CONCLUSION: Spectralis OCT retinal thickness measurements in healthy volunteers showed excellent intraobserver reproducibility with virtually identical results between retinal thickness measurements performed by one operator.

Accuracy and reproducibility of aortic annulus sizing using a dedicated three-dimensional computed tomography reconstruction tool in patients evaluated for transcatheter aortic valve replacement

Aims: To evaluate the accuracy and reproducibility of aortic annulus sizing using a multislice computed tomography (MSCT) based aortic root reconstruction tool compared with conventional imaging among patients evaluated for transcatheter aortic valve replacement (TAVR). Methods and results: Patients referred for TAVR underwent standard preprocedural assessment of aortic annulus parameters using MSCT, angiography and transoesophageal echocardiography (TEE). Three-dimensional (3-D) reconstruction of MSCT images of the aortic root was performed using 3mensio (3mensio Medical Imaging BV, Bilthoven, The Netherlands), allowing for semi-automated delineation of the annular plane and assessment of annulus perimeter, area, maximum, minimum and virtual diameters derived from area and perimeter (aVD and pVD). A total of 177 patients were enrolled. We observed a good inter-observer variability of 3-D reconstruction assessments with concordance coefficients for agreement of 0.91 (95% CI: 0.87-0.93) and 0.91 (0.88-0.94) for annulus perimeter and area assessments, respectively. 3-D derived pVD and aVD correlated very closely with a concordance coefficient of 0.97 (0.96-0.98) with a mean difference of 0.5±0.3 mm (pVD-aVD). 3-D derived pVD showed the best, but moderate concordance with diameters obtained from coronal MSCT (0.67, 0.56-0.75; 0.3±1.8 mm), and the lowest concordance with diameters obtained from TEE (0.42, 0.31-0.52; 1.9±1.9 mm). Conclusions: MSCT-based 3-D reconstruction of the aortic annulus using the 3mensio software enables accurate and reproducible assessment of aortic annulus dimensions.

Reproducibility of effects of homeopathically potentised gibberellic acid on the growth of Lemna gibba L. in a randomised and blinded bioassay

BACKGROUND: Reproducibility of basic research investigations in homeopathy is challenging. This study investigated if formerly observed effects of homeopathically potentised gibberellic acid (GA3) on growth of duckweed (Lemna gibba L.) were reproducible. METHODS: Duckweed was grown in potencies (14x-30x) of GA3 and one time succussed and unsuccussed water controls. Outcome parameter area-related growth rate was determined by a computerised image analysis system. Three series including five independent blinded and randomised potency experiments (PE) each were carried out. System stability was controlled by three series of five systematic negative control (SNC) experiments. Gibbosity (a specific growth state of L. gibba) was investigated as possibly essential factor for reactivity of L. gibba towards potentised GA3 in one series of potency and SNC experiments, respectively. RESULTS: Only in the third series with gibbous L. gibba L. we observed a significant effect (p = 0.009, F-test) of the homeopathic treatment. However, growth rate increased in contrast to the former study, and most biologically active potency levels differed. Variability in PE was lower than in SNC experiments. The stability of the experimental system was verified by the SNC experiments. CONCLUSIONS: Gibbosity seems to be a necessary condition for reactivity of L. gibba to potentised GA3. Further still unknown conditions seem to govern effect direction and the pattern of active and inactive potency levels. When designing new reproducibility studies, the physiological state of the test organism must be considered. Variability might be an interesting parameter to investigate effects of homeopathic remedies in basic research.

Reproducibility of peripheral arterial tonometry measurements in male cardiovascular patients

BACKGROUND Assessment of endothelial function of the microvasculature by peripheral arterial tonometry (EndoPAT(®)) has gained increasing popularity in patients with cardiovascular risk factors. Only limited knowledge about its reproducibility in patients with coronary artery disease (CAD) is available. We therefore aimed to quantify reproducibility of EndoPAT(®) parameters in patients with stable CAD. DESIGN EndoPAT(®) measurements were performed repeatedly in 78 male patients (age 66 ± 8 years) with CAD on stable medication. We calculated overall mean, standard deviation (SD), coefficient of variation (CV) and intraclass correlation coefficient (ICC) of the following parameters: reactive hyperemic index (RHI), PAT ratio of the postocclusion period 90-150 s as used for calculation of the RHI (PAT ratio90-150 s) and 90-120 s (PAT ratio90-120 s) as used for the often employed Framingham RHI (F-RHI), as well as PAT ratio of the peak hyperemic response (PAT ratiopeak response). Additionally, least significant changes (LSC) for individual subjects and minimum sample sizes for parallel and cross-over design studies were calculated. RESULTS Mean RHI was 1·84 (SD 0·36). For RHI, PAT ratio90-150 s , PAT ratio90-120 s , and PAT ratiopeak response the CVs were 17·0%, 25·4%, 26·1%, and 25·0%, respectively. The ICCs were 0·45, 0·49, 0·48 and 0·51, respectively, and LSC for RHI was 47·2%. CONCLUSIONS CV of RHI in our population was moderate; however, we consider this precision insufficient to monitor changes in individual patients, as they would need to exceed 47% to show a significant change. Further, the poor ICCs reflect the difficulty of detecting treatment effects in homogenous populations, such as patients with stable CAD.

Good reproducibility of heart rate variability after orthostatic challenge in patients with a history of acute coronary syndrome

BACKGROUND: Several parameters of heart rate variability (HRV) have been shown to predict the risk of sudden cardiac death (SCD) in cardiac patients. There is consensus that risk prediction is increased when measuring HRV during specific provocations such as orthostatic challenge. For the first time, we provide data on reproducibility of such a test in patients with a history of acute coronary syndrome. METHODS: Sixty male patients (65+/-8years) with a history of acute coronary syndrome on stable medication were included. HRV was measured in supine (5min) and standing (5min) position on 2 occasions separated by two weeks. For risk assessment relevant time-domain [standard deviation of all R-R intervals (SDNN) and root mean squared standard differences between adjacent R-R intervals (RMSSD)], frequency domain [low-frequency power (LF), high-frequency power (HF) and LF/HF power ratio] and short-term fractal scaling component (DF1) were computed. Absolute reproducibility was assessed with the standard errors of the mean (SEM) and 95% limits of random variation, and relative reproducibility by the intraclass correlation coefficient (ICC). RESULTS: We found comparable SEMs and ICCs in supine position and after an orthostatic challenge test. All ICCs were good to excellent (ICCs between 0.636 and 0.869). CONCLUSIONS: Reproducibility of HRV parameters during orthostatic challenge is good and comparable with supine position.

Reproducibility of retinal thickness measurements in patients with age-related macular degeneration using 3D Fourier-domain optical coherence tomography (OCT) (Topcon 3D-OCT 1000)

Conventional time-domain optical coherence tomography (OCT) has become an important tool for following dry or exudative age-related macular degeneration (AMD). Fourier-domain three-dimensional (3D) OCT was recently introduced. This study tested the reproducibility of 3D-OCT retinal thickness measurements in patients with dry and exudative AMD.

Reproducibility of sensory nerve conduction studies of the sural nerve using ultrasound-guided needle positioning

In this study we sought to evaluate the reproducibility of sensory nerve conduction studies (NCS) using ultrasound-guided needle positioning (USNP).